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Abstract:  The  non-linear  effects  that  have  heen  ohserved  in  adaptive  filtering  scenarios  are  explained  from  the  point  of 
view  of  the  structure  that  underlies  the  desired  data  While  the  model  structure  used  hy  the  conventional  adaptive  filter  is 
a  linear  combination  of  tappcd-dclay  line  signals,  that  adaptive  filter  model  docs  not  generally  correspond  to  the 
structure  that  best  describes  the  desired  data  one  is  adapting  to  The  nonlineai  effects  tn  adaptive  noise  canceling, 
interference  contaminated  adaptive  equalization,  and  adaptive  linear  prediction  arc  explained  here  as  being  the  result  of 
forcing  a  filter  model  onto  an  essentially  different  daia  struciurc.  The  tapped  delay  line  model  can  then  only  be 
compatible  with  the  data  if  the  filter  weights  become  time-varying.  If  the  adaptation  captures  the  lime-varying  weight 
behavior,  the  adaptive  filter  performance  can  approach  that  associated  with  the  data  structure  and  therchy  exceed  the 
best  performance  associated  with  the  corresponding  conventional  Wiener  filter. 


1,  INTRODUCTION 

Non-linear  effects  have  been  observed  in  several 
adaptive  filter  (AF)  applications,  usually  at  larger 
stepsizes  and  involving  narrowband  processes  Non¬ 
linear  effects  in  adaptive  noise  canceling  (ANC)  [1] 
interference  contaminated  adaptive  equalization  (AEQ) 
(2J,  and  adaptive  linear  prediction  (ALB)  [3]  have  been 
reported  and  analyzed  previously.  The  origin  of  the 
nonlinear  effects  in  ANC  was  studied  recently  (4J.  The 
explanations  depend  on  the  existence  of  an  underlying 
(Una  structure  that  can  produce  performance  well  in 
excess  of  the  performance  associated  with  the  optimal 
conventional  tapped-dclay  line  stmeuire  Wiener  tiller, 
which  is  time-invariant  in  the  reported  scenarios.  When 
ihe  lapped  delay  line  structure  is  used  in  an  AF  context 
its  weights  can  be  time-varying.  The  fundamental 
mechanism,  by  which  the  weights  adapt  to  the  data 
structure,  is  through  the  error  feedback,  as  the  error 
reflects  instantaneously  the  discrepancy  between  the 
structure  of  the  data  and  its  modeled  behavior.  T  he  AF 
may  capture  some  of  the  structure  that  underlies  the  data 
and  exhibit  improved  performance. 

2.  NLMS  ADAPTATION  &  MODELING 

The  nonlinear  effects  in  adaptive  filtering  were 
observed  when  using  the  least-mean  square  (I  MS) 
algorithm  and  its  normalized  (NLMS)  form  Wc  focus 
here  on  using  the  NLMS  algorithm,  in  which  the  error 
signal  is  computed  by  subtracting  the  desired  signal 
estimate  (the  AF  output,  computed  as  a  wei glued  linear 
combinaiion  of  all  its  inputs)  from  the  desired  signal 

e,  ~  <!„  -  <•/„  (1) 

d,  =  w”u,  (2) 

The  error  signal  is  used  to  correct  the  weight  vector. 


n»  <3> 

u'  u„ 

While  NLMS  can  he  applied  as  given,  its  properties 
depend  on  the  structure,  or  model,  for  the  desired  data. 
While  wc  know  the  model  that  underlies  the  estimate 
provided  by  the  adaptive  filter,  as  given  in  (2),  most 
often  we  do  not  know  the  structure  that  underlies  the 
desired  data.  A  common  assumption  is  that  the  desired 
signal  itself  has  the  same  linear  structure  as  that  used  for 
the  filter  estimate,  for  some  ideal  set  of  constanr 
weights,  vviih  perhaps  some  added  white  noise. 

dH  =  W*!!,  +  £„  (4) 

Under  these  assumptions  the  AF'  weights  converge  to  ihe 
true  weights,  or  a  neighborhood  about  the  true  weights 
determined  hy  the  additive  white  noise,  and  the  NLMS 
performance  -  in  terms  of  MSF,  approaches  the 
variance  of  the  additive  white  noise  from  above 
(depending  on  the  excess  MSE,  which  vanishes  as  the 
stepsize  parameter  JJ  vanishes).  The  structure  that 
underlies  the  data,  as  assumed  in  (4),  is  not  necessarily 
what  actually  generated  the  data.  If  the  model  assumed 
in  (2)  is  correct,  hut  its  order  is  too  low,  this  forces  part 
of  the  data  to  he  modeled  hy  the  additive  noise  and/or 
lime- varying  weight  behavior  If  the  model  assumed  in 
(2)  is  correct,  but  ihc  additive  noise  in  (4)  is  colored,  it 
is  possible  to  get  a  MSE  smaller  than  the  variance  of  the 
additive  noise  due  to  a  biased  parameterization.  So  far 
we  have  assumed  that  the  structure  underlying  the  data 
in  (4)  is  time-invariant  (Tl).  [f  the  true  weight  vector  in 
(4)  is  time-varying  (TV),  the  MSE  consists  of  a  tracking 
error  component  in  addition  to  the  estimation  error 
component. 

Any  TV  behavior  of  the  weight  in  (4)  is  immediately 
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reflected  in  the  desired  signal  and,  therefore,  in  the  error 
signal  At  large  stepsizes  this  effects  an  instantaneous 
change  in  the  a  posteriori  weight  vector  reflecting  the 
discrepancy  between  the  data  structure  (actual)  and  its 
tapped  delay  line  filter  model  (assumed).  We  show  that 
nonlinear  effects  can  arise  when  forcing  the  tapped 
delay  line  adaptive  filler  model  onto  more  general  data 
structures. 


3.  2-CHANNEL  NLMS 

The  input  vector  U„  used  in  (2)  usually  contains 

tappcd-dclay  line  values  based  on  a  single  reference 
channel.  For  the  purpose  of  revealing  underlying  data 
structures  we  assume  a  more  general  adaptive  filter 
structure,  where  lhe  input  vector  contains  delay  line 
values  from  the  reference  channel  as  well  as  from  the 
auxiliary  channel,  in  and  \fi  respectively 


11 


n 


(5) 


This  definition  renders  (l)-(3)  into  a  2 -channel  NLMS 
adaptive  filter,  as  reflected  in  Fig.  1 


Fig  f:  Basic  2  Channel  NLMS  Filter. 

In  the  conventional  NLMS  adaptive  filter,  the  input 
vector  does  not  contain  the  auxiliary  vector  partition 
directly,  hm  some  access  to  that  data  is  provided  in  the 
form  of  lhc  error  signal  (based  on  the  desired  data) 
which  is  used  in  the  weight  updates. 

■4.  VARIOUS  2-CHANNEL  SCENARIOS 

We  now  look  at  the  ANC,  AEQ,  and  ALP  scenarios 
from  the  2-channel  point  of  view,  Depending  on  the 
particular  sccnano,  the  auxiliary  channel  is  chosen  to 
reflect  the  underlying  structure  of  the  data.  Note  that 
NLMS  using  (5)  implies  lhe  data  can  be  represented  as 
in  (4),  using  lhe  2  channel  input.  The  lalier  has  been 
verified  by  applying  2-ehanncl  NLMS  and  observing 
that  both  the  weights  and  MSF  performance  approach 
those  of  the  corresponding  optimal  TI  Wiener  filter. 

4.1.  ANC 

In  the  conventional  ANC  scenario  the  desired  signal 
estimate  is  derived  from  present  and  past  values  of  the 


reference  signal.  It  is  beneficial  to  also  use  past  values 
of  the  desired  signal  f4].  Consequently,  the  appropriate 
2-chnnncl  ANC  scenario  uses  past  values  of  the  desired 
signal  for  the  auxiliary  channel,  as  reflected  in  Fig.  2. 


ltn 


Fig.  2.  2  Channel  ANC  Scenario. 


In  die  conventional  ANC  scenario,  the  reference  process 
carries  information  about  the  desired  process  because  of 
correlation.  In  the  2-channcl  ease  the  past  of  the  desired 
signal  is  similarly  expected  to  carry  additional 
information  about  its  immediate  future. 

4.2.  AKQ 

The  2  channel  AFQ  scenario  is  depicted  in  Fig.  3 


Fig  t  2-ChannH  AFQ  Scenario. 

In  the  conventional  mtcrfcrcncc-eontaminatcd  AEQ 
scenario,  the  adaptive  filter  derives  an  estimate  of  lhc 

training  sequence  sample  SM_n  from  the  reference 

rn  sn  +  in  +  )Jn  ,  an  interference  and  measurement 

noise  contaminated  signal.  Strong  narrowband 
interference  can  be  predicted  quite  well  from  its  past 
values,  so  that  having  the  interference  channel  as  the 
auxiliary  channel  could  he  beneficial. 

An  approximation  lo  this  2-channcl  structure  could  he 
implemented  by  using  the  reference  input  lo  provide  an 
estimate  for  the  interference  signal. 

4.3.  ALP 

In  the  conventional  adaptive  linear  predictor  lhe 
reference  input  consists  of  past  values  -  delayed  by  one 
or  more  samples  -  of  the  process  to  be  predicted  In  the 
2-channel  version  the  auxiliary  channel  consists  of  the 
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immediate  past  values  of  the  process  to  he  predicted 


Fig.  4.  2-Channel  A  I  P  Scenario. 

For  white  additive  noise  conditions  lhc  onc-stcp 
predictor  exhibits  the  best  performance,  but  when  the 
additive  noise  is  correlated,  it  can  be  desirable  to 
increase  the  delay  in  order  to  decorrelate  the  additive 
correlated  noise  component 
Recall  that  the  main  purpose  of  defining  the  above  2- 
channel  scenarios  is  to  reveal  the  structure  underlying 
the  desired  signal  in  each  case.  In  the  ANC  scenario,  the 
auxiliary'  channel  -  lhc  past  of  the  desired  signal  -  is 
actually  available  for  use  in  a  2-channel  approach.  In  the 
AFQ  and  Al  P  scenarios,  the  auxiliary  channel  -  the 
interference  signal  and  the  immediate  past  of  the  signal 
to  be  predicted,  respectively  -  is  not  measurable. 


5.  EQUIVALENT  CONVENTIONAL  FORM 

As  argued  in  Section  4,  we  start  with  the  lime- 
invariant  2-channcl  structure  for  the  desired  process 

dn  -  +  en 


f  .// 

// 1 

\n 

K 

W,  1 

—  ( l “  E  Cn 


When  the  conventional  adaptive  filter  is  used,  with 
only  the  reference  input  channel,  the  underlying  model 
to  be  estimated  is  of  the  following  resiricied  form 


cin  vv^r^  +eH 
-  ci  E  eM 

n  n 


0) 


To  see  what  happens  when  the  data  structure  that 
underlies  the  desired  process,  as  in  (6),  is  forced  onto 
the  model  used  in  conventional  NLMS,  as  in  (7),  we  use 
lhc  device  of  linking  sequences.  The  linking  sequence 

expresses  the  instantaneous  connection 


between  the  elements  Xm_m  and  t'M_{ ,  of  the  auxiliary 
and  reference  channel  respectively 


+  _  Nl  /-(-III+/)  /o\ 

Pn-I  -  “ 

rn-l  r»-t 

Consequently,  the  2-channel  structure  in  (6)  can  be 
rewritten  in  terms  of  the  conventional  model  of  (7).  For 
example,  supposing  that  the  auxiliary'  channel  has  a 
single  element  xu ,  tins  element  can  be  rewritten  in 

terms  of  the  reference  channel  element  r  as 
p\^]rFn_m  .  Using  this  substitution  in  (6)  yields: 


-kpM.,  +  wr)r,+e. 

where  1  m+1  is  an  indicator  vector  of  all  zeros  except  for 

a  I  as  the  (//j+l)st  element  Any  choice  for  the  latter 
location  is  equally  valid,  and  any  affine  linear 
combination  of  such  solutions  is  equally  valid.  The 
overall  linear  combinaiion  of  these  structures  descrihcs 
the  manifold  of  solutions  in  which  the  a  posieriori 
conventional  NLMS  weight  vector  resides 
Note  that  the  structure  in  (9),  rewritten  in  the  form  of 
the  conventional  structure,  has  the  same  -  usually  small 
additive  nnisc  as  the  data  structure  in  (6).  However, 
even  if  the  data  structure  in  (6)  has  T1  coefficients,  its 
conventional  model  equivalent  in  (9)  generally  has  time- 
varying  coefficients,  due  to  the  presence  of  the  linking 

sequence  .  In  the  manifold  a  linear  combination  of 

the  linking  sequences  }rt(;;0  would  be  present, 

where  A/  refers  to  the  total  number  of  elements  in  lhe 
reference  vector.  For  any  linear  combination  the  weight 
behavior  in  lhe  equivalent  conventional  struemre  is 
determined  by  the  behavior  of  the  linking  sequences.  As 
we  will  show  next,  the  specific  naiure  of  the  linking 
sequences  varies  with  the  application  scenarios  given  in 
Section  4. 


5.1.  ANC  Linking  Result 

In  the  ANC  application  for  which  nonlinear  effects 
have  hcen  reported  the  desired  and  reference  processes 
are  both  narrowband  AR{1),  and  centered  at  different 
frequencies  The  linking  sequences  behave  as  follow's. 


P 


») 

m 


PAi-n?,-! 

ps,  . ,  + 


(10) 


_  kyl/tw, -«,)(») 

w  p—  - 

Note  that  for  narrowband  processes  the  pole  radii  arc 
close  to  one,  so  that  away  from  the  zero-crossings  of 
the  reference  process  -  (10)  implies  that  (9)  contains  a 
weight  vccior  component  that  rotates  with  the  difference 
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frequency  of  the  desired  and  reference  processes.  In  xhe 
manifold  of  solutions  consecutive  elements  of  the  weight 
vector  contain  different  linking  sequences  that  are 
related  as  follows. 


OiM’V)  =  - 
Pn-mt I 


r  K-ni+l  /T-ffiVi 


P,r„.„  +  V„. 


(II) 


■\P'\~ 


la-p(”>  +v„ 


Note  that  neighboring  weights  arc  rotated  (with  random 
perturbation)  based  on  the  reference  process  center 
frequency.  These  weight  behaviors  correspond  to  the 
original  description  of  the  heterodyning  operation  [5] 


5.2.  AEQ  Linking  Result 

In  the  AHQ  application,  the  auxiliary  channel  consists 
of  in_D  and  the  reference  process  consists  of 

consecutive  elements  centered  about  fK^n  •  The  linking 
sequences  then  behave  as  follows. 


hy  dependence  on  NLMS  stepsize.  We  have  shown  that 
the  conventional  structure  equivalent  of  the  data  exhibits 
weight  behavior  that  is  nearly  deterministically  time- 
varying  (ANC)  or  subject  to  random  drift  (AEQ  and 
ALP)  NLMS  adaptation  implies  the  attempted  tracking 
of  the  structure  underlying  the  data  and,  depending  on 
the  measure  of  success  in  doing  so,  some  fraction  of  the 
potential  performance  gain  (determined  by  the  structure 
of  the  desired  data)  inay  be  realized. 

6.  CONCLUSIONS 

The  nonlinear  effects  that  occur  in  adaptive  filtering 
are  explained  by  investigating  what  happens  when  ihe 
conventional  tapped  delay  line  model  is  forced  upon  the 
multi-channel  structure  that  underlies  ihe  data.  When  the 
data  stmeture  does  not  correspond  to  a  time-invariant 
version  of  the  model  used  in  Ihe  adaptive  filter,  the 
constrained  structure  becomes  time-varying  and  an 
adaptive  filter  may  be  able  to  partially  track  that  time- 
varying  nature.  The  latter  explains  how  an  adaptive  filter 
can  outperform  tlie  corresponding  time-invariant  Wiener 
filter  when  both  use  the  conventional  tapped  delay  line 
model 


Pn-D+l 


P,  L-o* /  tfn-n 

*n  Oil  Oil  +fln  tut 


(12) 


Where  the  interference  process  dominates  the  noise  and 
signal  processes,  the  linking  sequence  produces  in 
neighboring  weights  a  rotation  with  magnitude  nearly 
one  and  phase  equal  to  the  center  frequency  of  the 
interference  process.  The  lime  dependent  behavior  of 
the  linking  sequence  lies  in  the  additive  noise  term  of 
the  last  line  in  (12),  so  that  the  TV  aspect  of  the 
corresponding  wcighi  in  (9)  is  a  random  drill 


5.3.  Al  P  Linking  Result 
The  ALP  scenario  linking  sequences  behave  as: 


P 


<'*•)  = 


r 


A  I  nr— I  / 

Pd  “n-A-m-rl 

//  4  ? 


I??, 

+  n 


~  (13) 

i  A  .-Jill 
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In  the  manifold,  the  consecutive  weight  vector  element 
portion  that  comes  front  projecting  the  interference 
signal  onto  the  reference  dimension  undergoes  a 
complex  rotation,  subjeci  to  additive  noise.  Only  ihe 
random  aspect  produces  time-varying  weight  behavior. 

The  nonlinear  effects  observed  in  ANC,  AEQ,  and 
ALP  [1-4]  are  cliaracterizcd  by  performance  that  is 
better  than  that  for  the  conventional  Wiener  filter,  and 
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